Computerized Evaluation of Essays

A " hot topic " in educational measurement is the area of computerized essay test evaluation. As essays have been incorporated into many of the standardized testing programs, and computerized testing is being implemented in more and more instances, there are many instances of where computers have evaluated the essay portion of standardized tests. In an environment where humans typically evaluate the submitted student writing, essay evaluations have been subject to variance in the level of agreement between readers and individual bias. The computer technology for essay evaluation has been made possible by the use of Naïve Bayes assumptions in text classification, that the probability of occurrence of each word in a document is independent of the probability of occurrence of other words in a document.. There are several computerized essay-scoring systems: (1) Project Essay Grade (PEG), (2) Latent Semantic Analysis (LSA), (3) Electronic Essay Rater (e-rater), (4) Bayesian Essay Test Scoring sYstem BETSY), and others. This paper discusses how these different scoring methods work, where they are currently implemented, and difficulties that have b een experienced in their development and implementation. The idea for classification essays by computer has been around for over thirty years, but the models have progressed from performing surface analysis only, content specific analysis, to a combination of both. They also utilize different statistical techniques, including linear regression models and factor analysis. Further improvements could be made when considering clusters and interactions of these features. These essay grading and classification systems could also be extended for use in the generation of writing diagnostics and instructional feedback.